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Abstract 

We define angles from-to and between infinite dimensional subspaces of a Hilbert space, in- 
spired by the work of E. J. Hannan, 1961/1962 for general canonical correlations of stochastic 
processes. The spectral theory of selfadjoint operators is used to investigate the properties of 
the angles, e.g., to establish connections between the angles corresponding to orthogonal com- 
plements. The classical gaps and angles of Dixmier and Friedrichs are characterized in terms 
of the angles. We introduce principal invariant subspaces and prove that they are connected by 
an isometry that appears in the polar decomposition of the product of corresponding orthogonal 
projectors. Point angles are defined by analogy with the point operator spectrum. We bound the 
Hausdorff distance between the sets of the squared cosines of the angles corresponding to the 
original subspaces and their perturbations. We show that the squared cosines of the angles from 
one subspace to another can be interpreted as Ritz values in the Rayleigh-Ritz method, where 
the former subspace serves as a trial subspace and the orthogonal projector of the latter subspace 
serves as an operator in the Rayleigh-Ritz method. The Hausdorff distance between the Ritz val- 
ues, corresponding to different trial subspaces, is shown to be bounded by a constant times the 
gap between the trial subspaces. We prove a similar eigenvalue perturbation bound that involves 
the gap squared. Finally, we consider the classical alternating projectors method and propose its 
ultimate acceleration, using the conjugate gradient approach. The corresponding convergence 
rate estimate is obtained in terms of the angles. We illustrate a possible acceleration for the do- 
main decomposition method with a small overlap for the ID diffusion equation. 
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1. Introduction 

Principal angles, also referred to as canonical angles, or simply as angles, between subspaces 
represent one of the classical mathematical tools with many applications. The cosines of the 
angles are related to canonical correlations which are widely used in statistics. Angles between 
finite dimensional subspaces have become so popular that they can be found even in linear alge- 
bra textbooks. 

The angles between subspaces T and Q are defined ■&& q - minjdim T , dim @] values on 
[0, 7r/2] if ^ < oo. In the case q - oo, where both subspaces T and Q are infinite dimensional, 
traditionally only single-valued angles are defined, which in the case q < oo would correspond 
to the smallest (Dixmier [11]), smallest non-zero (Friedrichs [13]), or largest (Krein et al. [29]), 
angles. We define angles from-to and between (infinite) dimensional subspaces of a Hilbert 
space using the spectra of the product of corresponding orthogonal projectors. The definition is 
consistent with the finite dimensional case q < oo and results in a set, possibly infinite, of angles. 

Our definition is inspired by E.J. Hannan [16], where such an approach to canonical corre- 
lations of stochastic processes is suggested. Canonical correlations for stochastic processes and 
functional data often involve infinite dimensional subspaces. This paper is intended to revive the 
interest in angles between infinite dimensional subspaces. 

In functional analysis, the gap and the minimum gap are important concepts used, e.g., in 
operator perturbation theory ([19]). The gap between infinite dimensional subspaces bounds 
the perturbation of a closed linear operator by measuring the change in its graph. We show in 
Theorem 2.12 that the gap is closely connected to the sine of the largest angle. 

The minimum gap between infinite dimensional subspaces provides a necessary and sufficient 
condition to determine if the sum of two subspaces is closed. The minimum gap is applied, 
e.g., in [22] to prove wellposedness of degenerate saddle point problems. The minimum gap is 
precisely, see Theorem 2.15, the sine of the angle of Friedrichs, which, in its turn, as shown in 
Theorem 2.14, is the infimum of the set of nonzero angles. The Dixmier angle is simply the 
smallest of all angles in our definition. 

We consider a (real or complex) Hilbert space equipped with an inner product (/, g) and a vec- 
tor norm ||/|| = (/, /)'^^. The angle between two unit vectors / and g is 0{f, g) = arccos |(/, g)\ € 
[Q,7t/2]. In §2 of the present paper, we replace ID subspaces spanned by the vectors / and g 
with (infinite dimensional) subspaces, and introduce the concept of principal angles from one 
subspace to another and between subspaces using the spectral theory of selfadjoint operators. 
We investigate the basic properties of the angles, which are already known for finite dimen- 
sional subspaces, see [23], e.g., we establish connections between the angles corresponding to 
subspaces and their orthogonal complements. We express classical quantities: the gap and the 
minimum gap between subspaces, in terms of the angles. 

In § 2, we provide a foundation and give necessary tools for the rest of the paper, see also [5] 
and references there. In § 3, we introduce principal invariant subspaces and prove that they are 
connected by the isometry that appears in the polar decomposition of the product of correspond- 
ing orthogonal projectors. We define point angles by analogy with the point operator spectrum 
and consider peculiar properties of the invariant subspaces corresponding to a point angle. In 
§ 4, the Hausdorff distance is used to measure the change in the principal angles, where one of 
the subspaces varies, extending some of our previous results of [23, 25] to infinite dimensional 
subspaces. 

We consider two applications of the angles: to bound the change in Ritz values, where the 
Rayleigh-Ritz method is applied to different infinite dimensional trial subspaces, in §5; and to 
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analyze and accelerate the convergence of the classical alternating projectors method (e.g., [10, 
Chapter IX]) in the context of a specific example — a domain decomposition method (DDM) 
with an overlap, in §6. In computer simulations the subspaces involved are evidently finite 
dimensional; however, the assumption of the finite dimensionality is sometimes irrelevant in 
theoretical analysis of the methods. 

In §5, we consider the Rayleigh-Ritz method for a bounded selfadjoint operator A on a trial 
subspace ;F of a Hilbert space, where the spectrum 'L{{PjrA)\jr) of the restriction to the subspace 
f of the product of the orthoprojector Pyr onto f and the operator A is called the set of Ritz 
values, corresponding to A and 'F. In the main result of §5, we bound the change in the Ritz 
values, where one trial subspace is replaced with another subspace 0, using the Hausdorff 
distance between the sets of Ritz values, by the spread of the spectrum times the gap between 
the subspaces. The proof of the general case is based on a specific case of one dimensional sub- 
spaces 'F and 0, spanned by unit vectors / and g, correspondingly, where the estimate becomes 
particularly simple: |(/, A/) - ig,Ag)\ < (^niax - '^min) sin(0(/,g)); here ^^ax - ■^mm is the spread 
of the spectrum of A, cf. [24]. If in addition / or g is an eigenvector of A, the same bound holds 
but with the sine squared — similarly, our Hausdorff distance bound involves the gap squared, 
assuming that one of the trial subspaces is A-invariant. The material of §5 generalizes some of 
the earlier results of [25, 26] and [27] for the finite dimensional case. The Rayleigh-Ritz method 
with infinite dimensional trial subspaces is used in the method of intermediate problems for de- 
termining two-sided bounds for eigenvalues, e.g., [36, 37]. The results of §5 may be useful in 
obtaining a priori estimates of the accuracy of the method of intermediate problems, but this is 
outside of the scope of the present paper 

Our other application, in § 6, is the classical alternating projectors method: e^'+'* - PjrPge^'\ 
g(0) g where f and ^ are two given subspaces and Pf- and Pg are the orthogonal projectors 

onto F and respectively. If j| (P^^f gr)|^j| < 1 then the sequence of vectors e*'' evidently 

converges to zero. Such a situation is typical if e*'' represents an error of an iterative method, e.g., 
a multiplicative DDM, so that the alternating projectors method describes the error propagation 
in the DDM, e.g., [38, 4]. 

If the intersection Ff^ff is nontrivial then the sequence of vectors e''' converges under reason- 
able assumptions to the orthogonal projection of e*"' onto Ti^Q as in the von Neumann-Halperin 
method, see [34, 15], and [2]. Several attempts to estimate and accelerate the convergence of al- 
ternating projectors method are made, e.g., [9, 2], and [39]. Here, we use a different approach, 
known in the DDM context, e.g., [38, 4], but apparently novel in the context of the von Neumann- 
Halperin method, and suggest the ultimate, conjugate gradient based, acceleration of the von 
Neumann-Halperin alternating projectors method. 

Our idea of the acceleration is inspired by the following facts. On the one hand, every self- 
adjoint non-negative non-expansion A, < A < / in a Hilbert space 'H can be extended to an 
orthogonal projector Pg in the space '7Y x Fl, e.g., [14, 31], and, thus, is unitarily equivalent to a 
product of two orthogonal projectors P-rPg restricted to the subspace T -Hy. jO). Any poly- 
nomial iterative method that involves as a main step a multiplication of a vector by A can thus 
be called an "alternating projectors" method. On the other hand, the conjugate gradient method 
is the optimal polynomial method for computing the null space of A, therefore the conjugate 
gradient approach provides the ultimate acceleration of the alternating projectors method. 

We give in §6 the corresponding convergence rate estimate in terms of the angles. We il- 
lustrate a possible acceleration for the DDM with a small overlap for the ID diffusion equation. 
The convergence of the classical alternating projectors method degrades when the overlap gets 
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smaller, but the conjugate gradient method we describe converges to the exact solution in two 
iterations. For a finite diff'erence approximation of the ID diffusion equation a similar result can 
be found in [12]. 

This paper is partially based on [18], where simple proofs that we skip here can be found. 

2. Definition and Properties of tlie Angles 

Here we define angles from one subspace to another and angles between subspaces, and 
investigate the properties of the (sets of) angles, such as the relationship concerning angles be- 
tween the subspaces and their orthogonal complements. We express the gap and the minimum 
gap between subspaces in terms of angles. We introduce principal invariant subspaces and prove 
that they are connected by an isometry that appears in the polar decomposition of the product 
of corresponding orthogonal projectors. We define point angles and their multiplicities by anal- 
ogy with the point operator spectrum, and consider peculiar properties of the invariant subspaces 
corresponding to a point angle. 

2.1. Preliminaries 

Let 'K be a (real or complex) Hilbert space and let T and Q be proper nontrivial subspaces. 
A subspace is defined as a closed linear manifold. Let P<jr and Pg be the orthogonal projectors 
onto T and respectively. We denote by SCH) the Banach space of bounded linear operators 
defined on with the induced norm. We use the same notation || - || for the vector norm on 'K, 
associated with the inner product (■, ■) on 'K, as well as for the induced operator norm on S('K). 
For T e SCH) we define |r| = yT*T, using the positive square root. T\u denotes the restriction 
of the operator T to its invariant subspace U. By ©(T), D\(T), ^{T), E(r), and l,p{T) we denote 
the domain, range, null space, spectrum, and point spectrum, respectively, of the operator T. In 
this paper, we distinguish only between finite and infinite dimensions. If <7 is a finite number 
then we set by definition min{^, 00} = q and max{q, 00) = 00, and assume that 00 < 00 holds. 
We use © to highlight that the sum of subspaces is orthogonal and for the corresponding sum of 
operators. We denote the operation between subspaces 'F and 0hyTQ0-Tr\ Q^. 

Introducing an orthogonal decomposition ^ - SJloo © 9^01 © 9^io © SRn © OT, where 

5Hoo = Tn^, OToi=5"n^^, mm^T^f^Q, Mu^T^ng^, 
(see, e.g., [14, 6]), we note that every subspace in the decomposition is Pjr and Pg invariant. 

Definition 2.1. (See [14]). Two subspaces T (I'M and CH are said to be in generic position 
within the space "H, if all four subspaces COToo, 93Toi, 9Jtio, fl«c/2tii are null-dimensional. 

Clearly, subspaces T '^'H and Q <^'H are in generic position within the space H iff any of 
the pairs of subspaces: 5^ c '7/ and c "H, oxT^ ^n< and Q ^'H,oxT^ and @^ c H, 
is in generic position within the space 'H. 

The fifth part, can be further orthogonally split in two different ways as follows: 

• m^mr® OTr^ with mr^TQ (OToo © 9Jloi), ^T^Q (Miq © OTh), or 

• m^mg® mg. with mg^ge mm © stIio), an^^ = © (SJioi © sotn). 
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We obtain orthoprojectors' decompositions 



Pt = -^OToo © ^OTo, © Oot,„ © Oot„ e Prb; and Pg = e Oot„, e e Oaii„ © Pg\m, 
and decompositions of their products: 

iPTPg)\r = /otoo © Ooto, e (Pr^g)lsat^, and {PgPT)\g = /otoo © Oot,„ © {PgPT)\me- 

These decompositions are very useful in the sequel. In the next theorem we apply them to prove 
the unitary equivalence of the operators PjrPgPfr and PgPyrPg. 

Theorem 2.2. Let T and Q he subspaces ofH. Then there exists a unitary operator W € SifH) 
such that PrPgPr = W'PgPrPgW. 

Proof. Denote T = PgPr- Then T* = PjrPg and T*T = P^PgPr- Using, e.g., [31, §110, 
p. 286] or [19, §VI.2.7, p. 334], we introduce the polar decomposition, T - U\T\, where 
|r| = ylT*T - ^P'jrPgP'jr is selfadjoint and nonnegative and U : — > $H(r) is an isometry. 
We extend U by continuity, keeping the same notation, to the isometry U : 5H(|r|) — > D\{T). It is 
easy to check directly that 9 ^(|r|) = m{T), so 9=^(|r|) = (9I(r))-^ since |r| is selfadjoint. Taking 
also into account that m(T) = (9^(7'))-^, we have U : C^iT))^ (9I(r*))-^. 

For a general operator T € SCTY), the isometry U is then typically extended to a partial 
isometry U G Si'H) by setting [/ = on 0^(7). For our special T = PgPr^ we can do better 
and extend f/ to a unitary operator W € SCH). Indeed, we setW = U on {^{T))-'- to make W 
an extension of U. To make W unitary, we set W = V on 9I(r), where V : ^{T) — » 9I(r*) 
must be an isometry. The specific form of V is of no importance, since it evidently does not 
affect the vaUdity of the formula PgPr - W ^JPrPgPr^ which implies PrPg - -^PrPePr^* ■ 
Multiplying these equalities we obtain the required PgPfPg - WPjrPgPjrW* . 

For the existence of such V, it is sufficient (and, in fact, necessary) that 91(r*) = ^(PjrPg) 
and Vl(T) - ^(PgPjr) be isomorphic. Using the five-parts decomposition, we get 

miPrPg) = moi®mm®mi®mPrPg)\m), "yiiPgPr) = moi®mo®mi®mPePr)\m)- 

The first three terms in the decompositions of ^(P'jrPg) and ^{PgPf) are the same, so ^(Pf-Pg) 
and m(PgPr) are isomorphic iff' the last terms m((PrPg)\m) = and WPgPr)\m) = 
are isomorphic. The subspaces DJlf- - Py^SDT c SOt and Tig - PgOJl c 9Jl are in generic 
position within the space 2Jl, see [14], as well as their orthogonal in 9JI complements dJljr± and 
dytg±. According to [14, Proof of Theorem 1, p. 382], any two subspaces in generic position are 
isomorphic, thus ^{PfPg) and ^{PgPjr) are isomorphic. □ 

Corollary 2.3. The operators {P'rPg)\mr '^^d. (PgP'r)\mg are unitarily equivalent. 

Proof. We have that PyrPgPyr = (P^Pg)|93!,^©/93!oo©0«e(OT„oeOT.r) and PgPr Pg = iPgPr)\mg® 
^ojioo ©0«e(OTooeOTis)- The subspaces ^Mf and dJlg are connected by Tlf = WDJlg, dJlg - W*'OJlr, 
and PrPgPr = WPgPrPgW. □ 

In the important particular case \\Pf- - Pg\\ < 1, subspaces and are isometric and Riesz 
and Sz.-Nagy [31, §VII.105] explicitly describe a partial isometry 

U = Pg[I + Pr(Pg - Pr)Prr"^Pr 
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that maps T one-to-one and onto Q. On T , clearly / -H P<f{Pg - Pjr)Pjr is just the same as 
PfPgPf, so this U represents the partial isometry in the polar decomposition in the proof of our 
Theorem 2.2, in this case. Let 

y - (/ - Pg)[I + (/ - PrW - Pg) -(I- PrM - PrW^Hl - Pr) 

be another partial isometry that maps 'F"'" one-to-one and onto 0-'-, constructed in the same way 
as U. Setting W = U + V, we extend U from the subspace 'F to a unitary operator W on the 
whole space. The sum W = U + V is the same as the unitary extension suggested in Kato [19, 
§1.4.6, §1.6.8] and Davis and Kahan [7]: 

W = [PgPr + (I-Ps)(I'Pr)]V-(Pr-Pgfr''^ (2.1) 
- [(/ - Pg)(I - Pr) + PTP6r"\PgPT + Pe)(I - Pr)] 

(the second equality holds since the corresponding terms in square brackets are the same and 
[Pf - Pg)~ commutes both with P^ and Pg), which is used there to prove the unitary equivalence 
Pr - W*PgW. It is easy to check directly that the operator W is unitary and that on 'F it acts the 
same as the operator U, so it is indeed a unitary extension of U. If \\Pr - PgW < 1, Theorem 2.2 
holds with this choice of W. 

In the next subsection we define angles from-to and between subspaces using the spectrum of 
the product of two orthogonal projectors. Our goal is to develop a theory of angles from-to and 
between subspaces based on the well-known spectral theory of selfadjoint bounded operators. 

2.2. Angles From-To and Angles Between Subspaces 

Definition 2.4. &(T, Q) ^ [6 : 6 ^ arccos(a-), cr > 0, cr- e Z{{P.rPg)\T)} £ [0, 7r/2] is called the 
set of angles from the subspace T to the subspace Q. Angles %{!T , Q) — ©{'F, Q) n &{Q, T) are 
called angles between the subspaces T and Q. 

Let the operator T e SCH) be a selfadjoint nonnegative contraction. Using an extension 
of T to an orthogonal projector [31, §A.2, p. 461], there exist subspaces 'F and in '7/- such 
that T is unitarily equivalent to {PyrPg)]^^ where Pf and Pg are the corresponding orthogonal 
projectors in 'H^. This implies that the spectrum of the product of two orthogonal projectors is 
as general a set as the spectrum of an arbitrary selfadjoint nonnegative contraction, so the set of 
angles between subspaces can be a sufficiently general subset of [0, n/l]. 

Definition 2.5. The angles @p(r,g) = {fle ©CT,^) : cos-(6i) e I.i,((PyrPg)\yr)]and&p(r,g) = 
^pCJ^^G) ^ ®p{@^'J^) ore called point angles. Angle e @p{'F,@) inherits its multiplicity from 
cos^(ff) 6 'Ep {{P'jrPg)\'jr). Multiplicity of angle 6 6 ®p{'F,0) is the minimum of multiplicities of 
6 e QpiT, 0) and e Qp(g, T). 

For two vectors / and g in the plane, and their orthogonal counterparts and g^ we evi- 
dently have that 0{f,g) = 0{f^,g^) and 0{f,g) + 0{f,g^) - n/2. We now describe relationships 
for angles, corresponding to subspaces F, Q, T^, and Q^. We first consider the angles from one 
subspace to another as they reveal the finer details and provide a foundation for statements on 
angles between subspaces. 

Theorem 2.6. For any pair of subspaces T and Q ofH: 

1. &{r,g^)^7ii2-&{r,g); 
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2. ®{g,T)\{nl2}^@{T,g)\{nl2}; 

3. &{r^,S) \ m U {nlD) = nil - mT,g) \ m U [nll])); 

4. ®{T^,g^) \ m U [nil]) = ©Cr,^) \ ({O} U {nll]); 

5. 0(^,:r^) \ {0) = Till - mr,9) \ {^/2)); 

6. 0(^^ T) \ [nlD = 7r/2 - {©CT, ^) \ {0)); 

7. 0(^^,5"^) \{0) = 0(5",^) \ {01. 



Table 1 : Multiplicities of and angles for different pairs of subspaces 



Pair 


6» = 




Pair 


6^0 


0^7t/2 




dim OToo 


dim OToi 




dim OToo 


dimOTio 




dimOToi 


dim OToo 




dimOTio 


dim OToo 




dimOTio 


dimOTii 




dimOToi 


dimOTii 




dim anil 


dim OJlio 




dimOTii 


dimOToi 



The multiplicities of the point angles e iO,n/2)in@(T,g), @{T^,Q^), @(Q,T) and®{g^,T^) 
are the same, and are equal to the multiplicities of the point angles 7T/2—0e (0, 7r/2) ;« ©CF, @'^), 
@{r^,Q), @{Q, r^) and @{@^, T). 

Proof (1) Using the equalities (_PrPg^)\T - P'F\r-{P'rPe)\'r - I\T-{PTPg)\r and the spectral 
mapping theorem for f{T) - I - T we have = 1 - E((PyrPg)|yr). Next, using the 

identity m(T - AI) ^ ^{{I - T) - (1 - X)I), we conclude that A is an eigenvalue of {PrP0)\r if 
and only if 1 - /I is an eigenvalue of {P'}rPg±)\jr, and that their multiplicities are the same. 
(2) The statement on nonzero angles follows from Corollary 2.3. The part concerning the zero 
angles follows from the fact that {PrPg)\^oa = (P0Pr)\mm = -^btoo- 

(3-7) All other statements can be obtained from the (1-2) by exchanging the subspaces. Table 1 
entries are checked directly using the five-parts decomposition. □ 

Theorem 2.7 and Table 2 relate the sets of angles between pairs of subspaces; 

Theorem 2.7. For any subspaces T and @ ofH the following equalities hold: 

1. ®{T,g) \ m U = {nil - 0(:r,^^)) \ ({0) U {7r/2)); 

2. 0(:r,^)\{O} = ©(5"\^^)\{O); 

3. ©(:r,^^)\{0) = ©(:r^,^)\{0). 

The multiplicities of the point angles in %{!J^ and &('F^,g^) satisfying < < 7t/2 are 
the same, and equal to the multiplicities of point angles < 7r/2 — < 7t/2 in ©{'FjQ^) and 

Proof. Statement (1) follows from Theorem 2.6 since 

0(:r,^)\({O)U{7r/2)) = @{T,Q)\m^{nl2]) 

- {7r/2-0(^,^^)}\({O)U{7r/2}) 
= {;r/2-0(ir,^^)}\({O)U{7r/2}), 
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Table 2: Multiplicities of and ;r/2 angles between subspaces 



Pair 


6» = 


0^7t/2 




dim OToo 


minjdimOToi, dimOTio) 




dimOToi 


minjdimOToo, dimOTn) 




dimOTio 


minjdimOToo, dimOTn) 




dimOTii 


minjdimOToi, dimOTio) 



Using Theorem 2.6(7) twice: first for f and 0, next for and f, and then intersecting them 
gives (2). Interchanging and 0-'- in (2) leads to (3). The statements on multiplicities easily 
follow from Theorem 2.6 as the entries in Table 2 are just the minima between pairs of the 
corresponding entries in Table 1 . □ 

Remark 2.8. Theorem 2.6(1) allows us to introduce an equivalent sine-based definition: 

®(r,g)^{0: = arcsin(ju), // > 0, ^i^ eI.{{PrPg^)\r)] ^ [0,^/2]. 

Remark 2.9. Theorem 2.6(2) implies @{T,g) \ {nil} = @{T,g) \ {n/l} = @(g,r) \ {tt/I}. 

Remark 2.10. We have 0(T,^) \ ({0) U {7r/2)) = ®{PmT, Pm&), in other words, the projec- 
tions PmT^ — '^nd Pm@ — of the initial subspaces T and Q onto their "fifth part" 
9Jl are in generic position within 9H, see [14], so the zero and right angles can not belong 
to the set of point angles QpiPmf^ ,Pmi@), but apart from and ;r/2 the angles and 
®{Pm'J^^ P<ynQ) ore the same. 

Remark 2.11. Tables 1 and 2 give the absolute values of the multiplicities ofO and n/2. If we 
need relative multiplicities, e.g., how many "extra" and n/2 values are in ©CF^ ,Q^) compared 
to we can easily find the answers from Tables 1 and 2 by subtraction, assuming that 

we subtract finite numbers, and use identities such as dimSTloo - dimOTn = dim'F — dim^-*" 
and dim^fftoi — dimSKio — dim'F — dim^. Indeed, for the particular question asked above, we 
observe that the multiplicity of n/2 is the same in ©{"F^ ,Q^) and in &{'F,§), but the difference 
in the multiplicities ofO in ©CF^ ,Q^) compared to in &{F, @) is equal to dim 9}li i — dim 9Jloo — 
dim^-"- - dimF, provided that the terms that participate in the subtractions are finite. Some 
comparisons require both the dimension and the codimension of a subspace to be finite, thus, 
effectively requiring dim'TY < oo. 

2.3. Known Quantities as Functions of Angles 

The gap bounds the perturbation of a closed linear operator by measuring the change in its 
graph, while the minimum gap between two subspaces determines if the sum of the subspaces 
is closed. We connect the gap and the minimum gap to the largest and to the nontrivial smallest 
principal angles. E.g., for subspaces F and in generic position, i.e., if 9Jl = 'H, we show that 
the gap and the minimum gap are the supremum and the infimum, correspondingly, of the sine 
of the set of angles between F and 0. 

The gap (aperture) between subspaces F and defined as, e.g., [19], 

gapiF,g) = ||Pr - Pg\\ = max {||PrP^x|| , \\PePr4} 
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is used to measure the distance between subspaces. We now describe the gap in terms of the 
angles. 

Theorem 2.12. min(min(cos-(0(:r,^))},min{cos2(0(^,:r))}} = 1 -geLp^{T,0). 

Proof. Let us consider both norms in the definition of the gap separately. Using Theorem 2.6, 
we have 

WPrPg-f = mp WPrPg^uf = sup {PrPg^u,PrPg^u) 

ue'H iie^H 
ll«ll=l ll«ll=l 

- sup (Pg.PrPe-u,u) = \\{Pg^Pr)\s-\\ = max{cos2(0(^^,O) 

ue'H 
ll«ll=l 

= max{sin2(0(^, T))} = 1 - min{cos2(0(^, T))}. 

Similarly, \\PgPrA\^ = max{cos"(0(:r^,^))) = 1 - min{cos2(0(:r,^))). □ 

It follows directly from the above proof and the previous section that 

Corollary 2.13. If gwpCF, @) < \ or if the subspaces are in generic position then both terms 
under the minimum are the same and so gap(;F, ^) — max{sin(0(;F, ^))). 

Let c{r,g) = sup{|(/,g)| ■.fere{TC\g\ ll/ll <l,g&Qe{Tr^g), \\g\\ < 1), as in [8], 
which is a definition of the cosine of the angle of Friedrichs . 

Theorem 2.14. In terms of the angles, c(T,@) = cos (inf {0(5^,^) \ {0})) . 

Proof. Replacing the vectors / = Pjru and g - Pgv in the definition of c('F, with the vectors 
u and V and using the standard equality of induced norms of an operator and the corresponding 
bilinear form, we get 

c{T,g)^ sup sup \{u,PrP0v)\^\\{PrP@)\'HemJV 

IMI=1 ll<il=l 

Using the five-parts decomposition, PrPg - ® Owioi ® ^^iw ® Ogj;,, © {PrPs)\m, thus 
"subtracting" the subspace SJtoo from the domain of PrPg excludes 1 from the point spectrum 
of PrPg, and, thus, from the set of point angles from T ioQ and, by Theorem 2.6(2), from the 
set of point angles between T and Q. □ 

Let the minimum gap, see [19, § IV.4], be defined as 

dist(f, Q) 



y{T,G)^ inf 



/er,/«^dist(f, 5^n^) 
Theorem 2.15. In terms of the angles, y{T, @) = sin (inf {®(T, @) \ {0))) . 
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Proof. We have f ^ f and f i 0, so we can represent / in the form f - fi + fi, where 
fiereimgifi^O and fiemg. Then 



,^ • f dist(f, ^) 



/er,/«g dist(f, 5^n^) 

Wfi+fi-Pgfi-Pefil 



inf 



fiere(rng).£ern@ \\fi + fo - Prngfi - PmgfiW 

- inf 

/,ere(rn^) - PrnffflW 

fereirng) \\f - PrngfW 

But / € (r n and 11/ - PrngfW = ll/IL Since |k/ - = - PgfW, using the 

Pythagorean theorem we have 

y{T,S) = mf — — 5 

inf 

/ere(rng), 11/11=1 

inf 1 - WPgff. 

feTe(Tng).\\f\\=l 

Using the equahty \\Pgf\\ = sup,,£g_||^||^i \{f,g)\ we get 

/(^F,^) = 1- sup \{f,gt 

fere(rr,g),geg,\\f\\=\\g\\=l 

= i-(c(r,^))' 

and finally we use Theorem 2. 14. □ 

Let us note that removing from the set of angles in Theorems 2.14 and 2.15 changes the 
result after taking the inf, only if is present as an isolated value in the set of angles, e.g., it has 
no effect for a pair of subspaces in generic position. 



2.4. The Spectra of Sum and Difference of Orthogonal Projectors 

Sums and differences of a pair of orthogonal projectors often appear in applications. Here, 
we describe their spectra in terms of the angles between the ranges of the projectors, which 
provides a geometrically intuitive and uniform framework to analyze the sums and differences of 
orthogonal projectors. First, we connect the spectra of the product and of the difference of two 
orthogonal projectors. 

Lemma 2.16. ([30, Theorem 1], [28, Lemma 2.4]). For proper subspaces T and Q we have 
Y(PrPg) = nPrPgPr) £ [0, 1] and 

nPg - Pr) \ ({-!) u {0) u {1)) = {±(1 - cr2)i/2 : e ^(PrPg) \ ({0) u {1))). 

Using Lemma 2.16, we now characterize the spectrum of the differences of two orthogonal 
projectors in terms of the angles between the corresponding subspaces. 
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Theorem 2.17. The multiplicity of the eigenvalue 1 in l.{Pg — Pf) is equal to dim9Jlio, the 
multiplicity of the eigenvalue —1 is equal to dimSUtoi, and the multiplicity of the eigenvalue is 
equal to dimOHoo + dimSJln, where OJloo, 2)Tob 9Hio and^Mn are defined in § 2.1. For the rest 
of the spectrum, we have the following: 

^(Pr - Pg) \ ({-!) U {0) U {!)) ^±sm(Q(r,g)) \ ({-1) U {0) U {!)). 

Proof. The last statement follows from Lemma 2.16 and Definition 2.4. To obtain the results 
concerning the multiplicity of eigenvalues 1,-1 and 0, it suffices to use the decomposition of 
these projectors into five parts, given in § 2. 1 . □ 

In some applications, e.g., in domain decomposition methods, see §6, the distribution of the 
spectrum of the sum of projectors is important. We directly reformulate [3, Corollary 4.9, p. 86], 
see also [33, p. 298], in terms of the angles between subspaces: 

Theorem 2.18. For any nontrivial pair of orthogonal projectors Pf- and Pg on the spectrum 
of the sum Pf- + Pg, with the possible exception of the point 0, lies in the closed interval of the 
real line [1 — H-P^r^'glL 1 + ll^'r^gll]> '^tnc/ the following identity holds: 

Y(Pr + Pg) \ ({0) u (D) = {1 ± cos(0(r,^))) \ ({0) u ID). 

3. Principal Vectors, Subspaces and Invariant Subspaces 

In this section, we basically follow Jujunashvili [18, Section 2.8] to introduce principal in- 
variant subspaces for a pair of subspaces by analogy with invariant subspaces of operators. Given 
the principal invariant subspaces (see Definition 3.1 below) of a pair of subspaces and 0, we 
construct the principal invariant subspaces for pairs 'F and 0-'-, and Q, and Q^. We 
describe relations between orthogonal projectors onto principal invariant subspaces. We show 
that, in particular cases, principal subspaces and principal vectors can be defined essentially as 
in the finite dimensional case, and we investigate their properties. Principal vectors, subspaces 
and principal invariant subspaces reveal the fine structure of the mutual position of a pair of 
subspaces in a Hilbert space. Except for Theorem 3.3, all other statements can be found in [18, 
sections 2.6-2.9], which we refer the reader to for detailed proofs and more facts. 

3.1. Principal Invariant Subspaces 

Principal invariant subspaces for a pair of subspaces generalize the already known notion 
of principal vectors, e.g., [35]. We give a geometrically intuitive definition of principal invariant 
subspaces and connect them with invariant subspaces of the product of the orthogonal projectors. 

Definition 3.1. A pair of subspaces H and "Y c is called a pair of principal invariant 
subspaces for the subspaces T and Q, if Pf'V C H and PgU £ 'V. We call the pair H 
and 'V Q Q nondegenerate if Pf'V = '1/ {0) and PgU - [Q] and strictly nondegenerate 
ifPjr'V = 1/ 5t {0} and Pg%l ^"V + {0). 

This definition is different from that used in [18, Section 2.8, p. 57], where only what we call 
here strictly nondegenerate principal invariant subspaces are defined. 

The following simple theorem deals with enclosed principal invariant subspaces. 
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Theorem 3.2. Let H <zT and <z Q be a pair of principal invariant subspaces for subspaces 
T and @, and 'U_ a 11, 'V_ <Z 'V be a pair of principal invariant subspaces for subspaces H 
and 'Y. Then 'U_, 'V_form a pair of principal invariant subspaces for the subspaces T , Q, and 

Definition 3.1 resembles the notion of invariant subspaces. The next theorem completely 
clarifies this connection for general principal invariant subspaces. 

Theorem 3.3. The subspaces H Q and 'Y c Q form a pair of principal invariant subspaces 
for the subspaces T and Q if and only if U Q is an invariant subspace of the operator 
{PrPg)\r and 'V = Pgll e 'V^, where "Vq c 97lio ^QC\T^. 

Proof Conditions P^'V c 11 and PgH c 'V imply PrPgK £ Pr^ £ 1^- Let us consider 
vo 6 e PgK - ^ nK-^ (the latter equaUty follows from = (vq, Pgu) - (vq, u), Vm g 11). 
We have Pj^vq e 1/-"- since K QT, but our assumption Pj^'V c H assures that P^rvc^ € H, so 
P-rVQ - 0, which means that 'Vq c OJlio, as required. 

To prove the converse, let PrPgK £ 1/ and 0^ = Pgll e ^o- Then P^^ = PrPgH £ H 
since K is closed. PgK £ ^ follows from the formula for □ 

If the subspace 9Hio is trivial, the principal invariant subspace ^ that corresponds to K is 
clearly unique. The corresponding statement for K, given 'V, we get from Theorem 3.3 by 
swapping f and 0. We now completely characterize (strictly) nondegenerate principal invariant 
subspaces using the corresponding angles. 

Theorem 3.4. The pair 11 QF and 'V Q @ of principal invariant subspaces for the subspaces 
T and Q is nondegenerate if and only if both operators {P'rPg)\u and {PgP'r)\'V are invertible, 
i.e., 71 12 i &p(H,'y) U &p('V,1l), and strictly nondegenerate if and only if each of the inverses 
is bounded, i.e., n/2 ( @{H,'V) U QCY^H), or equivalently in terms of the gap, gSLp(H,'V) — 

Wti - P-vll < 1- 

Proof. We prove the claim for the operator {PrPg)\v^ and the claim for the other operator fol- 
lows by symmetry. Definition 3.1 uses Pjr^ - K ^ [Q] for nondegenerate principal invariant 
subspaces. At the same time. Theorem 3.3 holds, so = PgH®'Vo, where 'Vq £ DJlio = QtyF^. 
SoK - P<j^'V - Py^PgK. Also by Theorem 3.3, K c F is an invariant subspace of the oper- 
ator {PjrPg)\r, soil - PrPgK - {PTPg)\nj'U- Since {PrPg)\ti is Hermitian, its null-space 
is trivial (as the orthogonal in K complement to its range which is dense in K), i.e., the oper- 
ator (P'rPg)\nj is one-to-one and thus invertible. For strictly nondegenerate principal invariant 
subspaces, {PyrPg)\tiK = K, so the operator {PrPg)\ii by the open mapping theorem has a 
continuous and thus bounded inverse. 

Conversely, by Theorem 3.3 K cF isan invariant subspace of the operator {PrPg)\r^ so the 
restriction {PrPg)\v is correctly defined. The operator {PrPg)\u is invertible by assumption, 
thus its null-space is trivial, and so its range is dense: K - {PjrPg)\tjK - P^Pgl^- By Theorem 
3.3, 'V = PgK e 'Vo, therefore Pr'V = PrPg^ = "i^- The other equaUty, PgK ^ + {0), 
of Definition 3.1 for nondegenerate principal invariant subspaces, is proved similarly using the 
assumption that (PgPT)\g is invertible. If, in addition, each of the inverses is bounded, the 
corresponding ranges are closed, 11 - PfPgll and 'V - P^Pg'V and we obtain P^'V -11 + 
{0) and PgK = (0) as is needed in Definition 3.1 for strictly nondegenerate principal 

invariant subspaces. 
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The equivalent formulations of conditions of the theorem in terms of the angles and the gap 
follow directly from Definitions 2.4 and 2.5 and Theorem 2.12. □ 



Theorem 2.2 introduces the unitary operator W that gives the unitary equivalence of PjrPgPjr 
and PgPjrPg and, if gap{f,0) < 1, the unitary equivalence by (2.1) of Pf- and Pg. Now we 
state that the same W makes orthogonal projectors P^ and P<y unitarily equivalent for strictly 
nondegenerate principal invariant subspaces U (zT and 'V <z Q, and we obtain expressions for 
the orthogonal projectors. 

Theorem 3.5. Let H Q T and 'V Q Q be a pair of strictly nondegenerate principal invariant 
subspaces for the subspaces T and Q, and W be defined as in Theorem 2.2. Then 'V — WU and 
II - W^, while the orthoprojectors satisfy P^ - WP^u^* - PgPt(((PrPe)\tlT^ Pt(P0 and 
Pv = WP^W = PrPv((PgPT)M-^PvPr- 

The proof of Theorem 3.5 is straightforward and can be found in [18, §2.8]. Jujunashvili 
[18, §2.9] also develops the theory of principal invariant subspaces, using the spectral decompo- 
sitions, e.g., below is [18, Theorem 2. 108]: 

Theorem 3.6. Let {Ei } and {E^} be spectral measures of the operators {P'rPg)\f and {PgPf)\g, 
respectively. Let C @{tl,'V) \ {tt/I} be a closed Borel set, and define P%i(@) — J^^^^^^dEi(A) 

and P'V{@) — Xos(0) ^P''i(^y Then U{&) C 'F and 'V(&) d @ is a pair of strictly nondegenerate 
principal invariant subspaces and 



and@ = = @i'V(®),^((®))■ 

Proof We have ^^^^^^ \dEi{A) = {{PrPv)\vr' = P« P« (where we denote 

U - 1/(0)), which we plug into the expression for the orthogonal projector P^y of Theorem 



For a pair of principal invariant subspaces H <zT and 'V (Z using Theorems 3.3 and 3.4 
we define the corresponding principal invariant subspaces in and @^ as l/j^ - P-jr±'V and 
= Pg^^tl, and describe their properties in the next theorem. 

Theorem 3.7. Let 11 and 'Y be a pair of principal invariant subspaces for subspaces T and Q 
and Q,nl2i 0(1/, 'V) U @{'V, tl). Then 1/^ = Pr^^ and ^ = Pgi^U are closed and 

• 1/ and 'V is a pair of strictly nondegenerate principal invariant subspaces for subspaces 



• l/j, and 'V is a pair of strictly nondegenerate principal invariant subspaces for subspaces 

and Q and P%i^ and Pry are unitarily equivalent; 

• H and 'y± is a pair of strictly nondegenerate principal invariant subspaces for subspaces 
T and and P<u and P^y^ are unitarily equivalent; 

• tl± and 'y± is a pair of strictly nondegenerate principal invariant subspaces for subspaces 

and and P<u^ and Pry^ are unitarily equivalent. 

Proof. The statements follow directly from Theorems 3.3 and 3.4 applied to the corresponding 
pairs of subspaces. The closedness of tlx. and 'V ^ can be alternatively derived from Theorem 




3.5. 



□ 



T and Q; 



2.14 and [8, Theorem 22]. 



□ 
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3.2. Principal Subspaces and Principal Vectors 

For a pair of principal invariant subspaces U izT and 'V <z Q, '\f the spectrum l.{{P<jrPg)\ii) 
consists of one number, which belongs to (0, 1] and which we denote by cos^(0), we can use 
Theorem 3.5 to define a pair of principal subspaces corresponding to an angle 9: 

Definition 3.8. Let 6 e ®{T,Q) \ {7t/2}. Nontrivial subspaces U Q T and 'V Q Q define a 
pair of principal subspaces for subspaces T and Q corresponding to the angle if {,P'J^P'v)\t — 
cos?{6)Pii and(PgP%()\g - cos^{6)P'y. Normalized vectors u = u(0) € 'F andv - v(0) € Qform 
a pair of principal vectors for subspaces T and Q corresponding to the angle ifPy^v — cos(0)u 
and Pgu — cos(0)v. 

We exclude - 7r/2 in Definition 3.8 so that principal subspaces belong to the class of 
strictly nondegenerate principal invariant subspaces. We describe the main properties of principal 
subspaces and principal vectors that can be checked directly (for details, see [18]). The first 
property characterizes principal subspaces as eigenspaces of the products of the corresponding 
projectors. 

Tlieorem 3.9. Subspaces H <zT and 'V c Q form a pair of principal subspaces for subspaces 
T and Q corresponding to the angle 6 %(!T , @) \ [nl2] if and only if0 e Qpi'F, Q) \ {n/l} andU 
and "V are the eigenspaces of the operators {P'^Pg)\j=' and (_PgP'r)\g, respectively, corresponding 
to the eigenvalue cos^{0). In such a case, @{'U,'V) — @p{'U,'V) — {0}. All pairs of principal 
vectors u and v of subspaces T and Q corresponding to the angle generate the largest principal 
subspaces U and 'V corresponding to the angle 0. 

Tlieorem 3.10. Let 1/(0), 1({ip) c T, and 'V{0), 'V{(p) (Z g be the principal subspaces for 
subspaces T and Q corresponding to the angles 0,(p & ®p{'F,Q) \ {7r/2). Then P'U{g)Pu(^) — 
PwB)nw^); P'V(0)P'V(4>} - P'Vim'Vi,/,) ; Pwe) and P'V(^) are mutually orthogonal if0^(f)(if0-4> 
we can choose 'V(0) such that P'U(8)Pv(e) — Pwe)P@)> for given U{0) we can choose 'V{0) such 
that P'V(e)P%i{e) = Pv{e)Pr- 

Corollary 3.11. [of Theorem 3. 7] Let 11 and 'V be the principal subspaces for subspaces T and 
g, corresponding to the angle e ®p{T,@) \ ({0) U {nil}). Then 1/^ = Pjr^'V and'V^ = Pg^nj 
are closed and 

• U^, 'Y are the principal subspaces for subspaces and Q, corresponding to the angle 
nil- 0; 

• H, 'V ^ are the principal subspaces for subspaces T and Q^, corresponding to the angle 
nil - 0; 

• H^, "Vj. are the principal subspaces for subspaces andQ^, corresponding to the angle 
0. 

Let u and v form a pair of principal vectors for the subspaces T and Q, corresponding to the 
angle 0. Then u± — (v — cos(0)m)/ sin(0) and Vj_ — (u — cos(0)v)/ sin(0) together with u and v 
describe the pairs of principal vectors. 



14 



4. Bounding the Changes in the Angles 

Here we prove bounds on the change in the (squared cosines of the) angles from one subspace 
to another where the subspaces change. These bounds allow one to estimate the sensitivity of the 
angles with respect to the changes in the subspaces. For the finite dimensional case, such bounds 
are known, e.g., [23, 24]. To measure the distance between two bounded real sets S i and 52 we 
use the Hausdorff distance, e.g., [19], dist(Si,S2) = maxlsupygj^ dist(u, S2), sup^^j^ dist(v. Si)), 
where dist(u, S) = infyes |u - v| is the distance from the point u to the set S . The following 
theorem estimates the proximity of the set of squares of cosines of QCF, Q) and the set of squares 
of cosines of ©{T, Q), where T, Q and Q are nontrivial subspaces of H. 

Theorem 4.1. dist(cos2(0(5^,^)),cos2(0(5^,^))) < gap(^,^). 

Proof. cos^{&{T,g)) = I.{{PrP@)\r) and cos^iei'F,^)) = I.{{PrPg)\r) by Definition 2.4. 
Both operators {PrPg)\r and {PrPg)\r are selfadjoint. By [19, Theorem 4.10, p. 291], 

dism(PrPg)\r),mPrPg)\r)) < \\(PrPg)\r - (PrP@)\r\l 

Then, \\(PrPg)\r - (PrPe)\r\\ <\\PrmPg- PgWWPrW < gap(^, □ 

The same result holds also if the first subspace, 'F, is changed in 0('F, 0)): 
Theorem 4.2. dist(cos2(0(5^, g)), cos~(@(f, &))) < gapCT, f). 

Proof. The statement of the theorem immediately follows from Theorem 5.2, which is indepen- 
dently proved in the next section, where one takes A = Pg. □ 

We conjecture that similar generalizations to the case of infinite dimensional subspaces can 
be made for bounds involving changes in the sines and cosines (without squares) of the angles 
extending known bounds [23, 24] for the finite dimensional case. 

5. Changes in the Ritz Values and Rayleigh-Ritz error bounds 

Here we estimate how Ritz values of a selfadjoint operator change with the change of a 
vector, and then we extend this result to estimate the change of Ritz values with the change of a 
(infinite dimensional) trial subspace, using the gap between subspaces, gap('F, ^) - \\Pf- - Pg\\. 
Such results are natural extensions of the results of the previous section that bound the change 
in the squared cosines or sines of the angles, since in the particular case where the selfadjoint 
operator is an orthogonal projector its Ritz values are exactly the squared cosines of the angles 
from the trial subspace of the Rayleigh-Ritz method to the range of the orthogonal projector. In 
addition, we prove a spectrum error bound that characterizes the change in the Ritz values for 
an invariant subspace, and naturally involves the gap squared; see [27, 1, 26] for similar finite 
dimensional results. 

Let A e SCH) be a selfadjoint operator Denote by A{f) - (f,Af)l{f,f) the Rayleigh 
quotient of an operator A at a vector f + Q. In the following lemma, we estimate changes in the 
Rayleigh quotient with the change in a vector. This estimate has been previously proven only 
for real finite dimensional spaces [24]. Here, we give a new proof that works both for real and 
complex spaces. 
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Lemma 5.1. Let A 6 SCH) be a selfadjoint operator on a Hilbert space and f,g € 'H with 
f,g ^ 0. Then 

\A{f) - A{g)\ < (max{S(A)) - min{2(A)})sin(0(/,g)). (5.1) 

Proof. We use the so-called "mini-dimensional" analysis, e.g., [20, 21]. Let S - span{f, g) c '7Y 
be a two dimensional subspace (if / and g are linearly dependent then the Rayleigh quotients 
are the same and the assertion is trivial). Denote A - {PsA)\s and two eigenvalues of A by 
A\ < /I2. By well known properties of the Rayleigh-Ritz method, we have A{f), A{g) € [/li, /I2] £ 
[max{S(A)), min{2(A))]. In the nontrivial case A{f) + A{g), we then have the strong inequality 
A\ < A2. 

In this proof, we extend the notation of the Rayleigh quotient of an operator A at a vector / 
to A{f \ A) - (f,Af)/(f,f) to explicitly include A. It is easy to see that A{f; A) = A(f; A) and 
that the same holds for vector g. Then, since [/li, /I2] £ [max{E(A)), min{S(A))] the statement of 
the lemma would follow from the 2D estimate \A(f; A) - A(g; A)| < (A2 - Ai) sm(6(f,g)) that 
we now have to prove. The latter estimate is clearly invariant with respect to a shift and scaling 
of A. Let us use the transformation A = (A - AiI)/{A2 - Ai) then the estimate we need to prove 
turns into \A{f; A) - A{g; A)| < sin(6'(/, g)), but the operator A has two eigenvalues, zero and one, 
and thus is an orthoprojector on some one dimensional subspace spanjh) c S. Finally, A{f; A) - 
(f,Pi,f)/(fJ) = cos\0(h,f )) and, similarly, A{g; A) = (g, P,,g)/(g, g) = cosH0(h,g)). But 
I cos2(0(/z, /)) - cosHOih, g))\ = I \\P,,PfP,,\\ - WPhPsPhW I < \\Pf - PgW = sin(0(/, g)). □ 

In the Rayleigh-Ritz method for a selfadjoint operator A € SCN) on a trial subspace 'F 
the spectrum 2((PyrA)|yr) is called the set of Ritz values, corresponding to A and 'F. The next 
result of this section is an estimate of a change in the Ritz values, where one trial subspace, 
F, is replaced with another, 0. For finite dimensional subspaces such a result is obtained in 
[24], where the maximal distance between pairs of individually ordered Ritz values is used to 
measure the change in the Ritz values. Here, the trial subspaces may be infinite dimensional, 
so the Ritz values may form rather general sets on the real interval [min{2(A)), max{S(A))] and 
we are limited to the use of the HausdorfF distance between the sets, which does not take into 
account the ordering and multiplicities. 

Theorem 5.2. Let A € SCH) be a selfadjoint operator and T and Q be nontrivial subspaces of 
'H. Then a bound for the Hausdorff distance between the Ritz values of A, with respect to the 
trial subspaces T andQ, is given by the following inequality 

dist(2((PrA)|^), S((PgA)|^)) < (max{S(A)) - min{E(A)))gap(5",^). 

Proof. If gap('F, 0) - I then the assertion holds since the both spectra are subsets of [min{2)(A)), max{E(A))] . 
Consequently we can assume without loss of generality that gap('F, ^) < 1. Then we have 
g = WT with W defined by (2.1). Operators {PgA)\@ and {W*{PgA)\gW) \r are unitarily equiv- 
alent, since W is an isometry on T, therefore, their spectra are the same. Operators {PjrA)\'jr and 
(W*(PgA)\gW) If- are selfadjoint on the space F and using [19, Theorem 4.10, p. 291] we get 

dist(2((PrA)|^), E((PgA)|g)) = dist(S((PrA)|^), Y(iW*(PgA)\gW)\r)) 

< UPrA-W*{PgA)\gW)\r\\. (5.2) 



Then 



\{PrA-W*{PgA)\gW)\r\\ = sup \{{PrA-W*(PgA)\gW)f,f)\ 

11/11=1, /er 

sup \(Af,f)-(AWf,Wf)\. 

11/11=1, /er 
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We have \{f,Af) - (Wf,AWf)\ < (max{S(A)) - min{2(A)}) Vl - l(/, V/ e T, ||/|| = 1 by 

Lemma 5.1. We need to estimate |(/, Wf)\ from below. From the polar decomposition PgPr — 
^ ^JPtPsPt^ we derive the equalities 

(/, Wf) = {PgPrf, Wf) = {WPgPrfJ) = ( y/PrPgPrfJ), 

where we have -s/PrPgPrlr - '^iPrP0Pr)\r - '\J(PrPe)\r^ since 'F is an invariant subspace 
of the operator PrPgPr- Thus, (/, Wf) = ( yJ{PrPe)\rfJ) > min{cos(©(5^,^))) by Definition 
2.4. Finally, by assumption, gap(;F, ^) < 1, thus Corollary 2.13 gives min{cos^(0(!F, ^))} - 
\-gap\T,Q). □ 

Finally, we assume that T is A-invariant, which implies that the set of the values ^A)|^) 
is a subset, namely 2(A|^), of the spectrum of A. The change in the Ritz values, bounded in 
Theorem 5.2, can now be interpreted as a spectrum error in the Rayleigh-Ritz method. The result 
of Theorem 5.2 here is improved since the new bound involves the gap squared as in [1, 26]. 

Theorem 5.3. Under the assumptions of Theorem 5.2 let in addition T be an A-invariant sub- 
space ofH corresponding to the top (or bottom) part of the spectrum of A. Then 

dist(2(A|^), 2((PgA)|g)) < (max{2(A)) - min{2(A)))gap2(:r,^). 

Proof. As the subspace T is A-invariant and A is selfadjoint, the subspace is also A-invariant, 
so A = PyrAPyr H- Pjr^APjr± and, with a slight abuse of the notation, A = Alj^ -h A|^±, correspond- 
ing to the decomposition 1-1 - T ® T^, thus S(A) = £(A|^) U S(A|^x). We assume that T 
corresponds to the top part of the spectrum of A — the bottom part case can be treated by replac- 
ing A with -A. Under this assumption, we have max{E(A|yrx)) < min{Z(A|^)). 

Let us also notice that the inequality we want to prove is unaltered by replacing A with A-al 
where a is an arbitrary real constant. Later in the proof we need A|^ to be nonnegative. We set 
a - min{2(A|yr)) and substitute A with A - al, so now max{Z(A|yr±)) < = min{2(A|3r)), thus 

IIAIrll = max{I(A|^)) = max{2(A)), and \\A\rA\ = -vam{L{A\r^)} = -min{S(A)). 

The constant in the bound we are proving then takes the following form: 

max{S(A)) - min{2(A)) = ||A|^|| + ||A|^x|| . (5.3) 

As in the proof of Theorem 5.2, if gap(;F, ^) - 1 then the assertion holds since the both 
spectra are subsets of [min{S(A)), max{E(A))]. Consequently we can assume without loss of 
generality that gap(;F, ^) < 1. Then we have Q - WT with W defined by (2.1). Operators 
{W*{PgPyrAPf)\g'W) \ f wA {PgPfAPf)\g arc unitarily equivalent, since W is an isometry on 
T, thus their spectra are the same. Now, instead of (5.2), we use the triangle inequality for the 
Hausdorff distance: 

dist(Z(A|^), S((P^A)|g)) 

< dist(Z((A|r), S((W*(PgP^AP^)|gW) \r)) 

+ dist(2((PgPrAPr)|j^)), ^{(¥@K)\@)). (5.4) 

The operator -\/PrPgPr\r - ■\/(PtPs)\t is selfadjoint and its smallest point of the spectrum 
is min{cos(0(!F, ^))) by Definition 2.4, which is positive by Theorem 2.12 with gap(;F, ^) < 1. 
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The operator -\/PrP0Pr\r is invertible, so from the polar decomposition PgPr - W ^jPfPgPf, 
which gives PyrPgPjr = PjrPgW iJPpPgPp, we obtain by applying the inverse on the right that 
{PrPgW)\r = ^JPrPgPr\r = {W*PgPr)\T- Thus, 

(WHP@PrAPr)\eW)\r = {^[P^P^A 4P^P^\r 

= ^/P^Pp^\r ^IA\^ ^/A\^ ^/P^P^\^ 

where the operator A|yr is already made nonnegative by applying the shift and the substitution. 

The spectrum of the product of two bounded operators, one of which is bijective, does not 
depend on the order of the multipliers, since both products are similar to each other One of our 
operators, ^PfrPgP'r\r, in the product is bijective, so 

I.{{W\PgPrAPr)\gW) \r) = 1.{4a\^{PtP9)\t V^lr) • 

Then the first term in the triangle inequality (5.4) for the Hausdorff distance is estimated using 
[19, Theorem 4. 10, p. 291]: 

dist(2 {A\r) , I ((W* (PgPrAPr) \gW) \r) ) 

= dist (Z (A\r) , 2 ( V^(PrPg)lr V^) 

< \\A\r- ^^A\^(PrPe)\r^^A\^\\ 
= \\^^A\^(Pr - PrPg)\r ^^A\^\\ 

< \\A\r\\ \\(PrPg-)\r\\ = ||^r^^-||' ■ 
To estimate the second term in (5.4), we apply again [19, Theorem 4. 10, p. 291]: 

dist(5: i(PgPrAPr)\g)), WPgA)\g)) < \\(PgPrAPr)\g - (PgA)|g|| 

= \\(PgPr-APr^)\g\\ = WPgPr^Alr^Pr-PgW < IIA|r-llll^g^r-ll', 
where A - PyrAPyr + Pjr±APjr± . Plugging in bounds for both terms in (5.4) gives 

dist(2(A|r), mPeA)\g)) < \\A\r\\ \\PrPg4' + ||PeP^-||' • 

Assumption gap{r,0) < 1 implies that H^'r^'^^ = \\P@PrA\ = gap(r,g), e.g., see [19, §1.8, 
Theorem 6.34] and cf. Corollary 2.13. Thus we obtain 

distmPrA)\r), WPgA)\g)) < (\\A\r\\ + \\A\r4)g^pHr,g). 

Taking into account (5.3) completes the proof. □ 

We conjecture that our assumption on the invariant subspace representing a specific part of 
the spectrum of A is irrelevant, i.e., the statement of Theorem 5.3 holds without it as well, cf. 
Argentati et al. [1], Knyazev and Argentati [26]. 
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6. The ultimate acceleration of the alternating projectors method 

Every selfadjoint nonnegative non-expansion A, < A < / in a Hilbert space "TY can be 
extended to an orthogonal projector in the space 'Hx'H, e.g., [14, 31], and, thus, can be implicitly 
written as (strictly speaking is unitarily equivalent to) a product of two orthogonal projectors 
PrPg restricted to a subspace f c x . Any iterative method that involves as a main step a 
multiplication of a vector by A can thus be called "an alternating projectors" method. 

In the classical alternating projectors method, it is assumed that the projectors are given 
explicitly and that the iterating procedure is trivially 

If II (Py^Pg^)]^! < 1 then the sequence of vectors e*'' evidently converges to zero. Such a situation 

is typical when e^'' represents an error of an iterative method, e.g., in a multiplicative DDM, and 
formula (6. 1) describes the error propagation as in our DDM example below. 

If the subspace SlJloo = ;F n ^ is nontrivial and || (^r^g)|^Qrj^ || ^ ^^^^ sequence of 
vectors e*'* converges to the orthogonal projection e of e*°^ onto IDtoo- The latter is called a von 
Neumann-Halperin ([34, 15]) method in [2] of alternating projectors for determining the best 
approximation to e*"^ in 2Hoo- We note that, despite the non-symmetric appearance of the error 
propagation operator P-rPg in (6.1), it can be equivalently replaced with the selfadjoint operator 
{PrPg)\jr since e^"* 6 T and thus all € T. 

Several attempts to estimate and accelerate the convergence of iterations (6. 1) are made, e.g., 
[9, 2, 39]. Here, we use a different approach, cf., e.g., [38, 4], to suggest the ultimate acceleration 
of the alternating projectors method. First, we notice that the limit vector e e SJIqo is a nontrivial 
solution of the following homogeneous equation 

{I-PrPg)\^e^Q, eer. (6.2) 

Second, we observe that the linear operator is selfadjoint and nonnegative in the equation above, 
therefore, a conjugate gradient (CG) method can be used to calculate approximations to the 
solution e in the null-space. The standard CG algorithm for linear systems Ax - b can be 
formulated as follows, see, e.g., [17]: 

Initialization: set y = 1 and compute the initial residual r - b - Ax; 
Loop until convergence: 
Joid ^7,7^ (r, r); 

on the first iteration: p - r\ otherwise: 

P = yljoid (standard) or/? = (r - r^M, r)/{r,M, rdd) 

(the latter is recommended if an approximate application of A is used) 

p - r + Pp, r — Ap, a — yl{r,p), x — x + ap, r — r — ar. 

End loop 

It can be applied directly to the homogeneous equation Ae = with A = A* > by setting 
= 0. We need A - {I - PfPg^^ for equation (6.2). Finally, we note that CG acceleration 
can evidently be applied to the symmetrized alternating projectors method with more than two 
projectors. 

The traditional theory of the CG method for non-homogeneous equations extends trivially to 
the computation of the null-space of a selfadjoint nonnegative operator A and gives the following 
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convergence rate estimate: 



(e^'^KAe^''^) < min sup \pk(A)f (e^°\Ae^°^). (6.3) 

degpt=k,pt(Q)=i Ael.(A)\{Q} 

For equation (6.2), A = (/- P?.Pg)|^ and thus (eW,AeW) = ||PgxeW||2 ^nd by Definition 2.4 we 
have X(A) = 1 -cos^ ©CF, 0)- Estimate (6.3) shows convergence if and only if zero is an isolated 
point of the spectrum of A, or, in terms of the angles, if and only if zero is an isolated point, or 
not present, in the set of angles ©{T , Q), which is the same as the condition for convergence of 
the original alternating projectors method (6.1), stated above. 

Method (6.1) can be equivalently reformulated as a simple Richardson iteration 

gW = (/ - A)^e<'", e<°> 6 T, where A = (/ - Pr-Pg)|^ > 

and thus falls into the same class of polynomial methods as does the CG method. It is well 
known that the CG method provides the smallest value of the energy (semi-) norm of the error, 
in our case of HP^ie**^*!!, where e*'^-' e T, which gives us an opportunity to call it the "ultimate 
acceleration" of the alternating projectors method. 
A possible alternative to equation (6.2) is 

(P^x +Pgx)e = 0, (6.4) 

so we can take A = + Pg± in the CG method for equation (6.4) and then Z(A) is given 
by Theorem 2.18. Equation (6.4) appears in the so-called additive DDM method, e.g., [32]. A 
discussion of (6.4) can be found in [18, §7.1, p. 127]. 

Estimate (6.3) guarantees the finite convergence of the CG method if the spectrum of A 
consists of a finite number of points. At the same time, the convergence of the Richardson 
method can be slow in such a case, so that the CG acceleration is particularly noticeable. In 
the remainder of the section, we present a simple domain decomposition example for the one 
dimensional diffusion equation. 

Consider the following one dimensional diffusion equation u'v'dx - fv'dx, Vv € 
^^o([0, 1]) with the solution u € //^([0, 1]), where i/g([0, 1]) is the usual Sobolev space of real- 
valued functions with the Lebesgue integrable squares of the first generalized derivatives and 
with zero values at the end points of the interval [0, 1]. We use the bilinear form u'v'dx as a 
scalar product on //g([0, 1]). 

We consider DDM with an overlap, i.e., we spHt [0, 1] = [0, a] U ^6, 1], with < j6 < a < 1 
so that [/6, a] is an overlap. We directly define orthogonal complements: 

^{ue //^([0, 1]) : u{x) ^Q,xe [a, 1]) and^-" = {v € H^dQ, 1]) : v{x) ^Q,xe [0,^6]) 

of subspaces T c //^([0, 1]) and Q c Hl([Q, 1]). Evidendy, 'H ^T^+ where the sum is not 
direct due to the overlap. 

It can be checked easily that the subspace T consists of functions, which are linear on the 
interval [0, a] and the subspace Q consists of functions, which are linear on the interval [/6, 1]. 
Because of the overlap \J3,a\, the intersection 9Jloo - T C\ Q is trivial and the only solution of 
(6.2) and (6.4) is e = 0. 

We now completely characterize all angles between T and Q. Let / e ;F be linear on intervals 
[0, a] and [a, I]. Similarly, let § e ^ be linear on intervals [0,j6] and [/6, 1]. It is easy to see, 
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cf. [18, §7.2], that all functions in the subspace T span{f) vanish outside of the interval \a, 1], 
while all functions in the subspace ^espan{g) vanish outside of the interval [0,/?]. Therefore, the 
subspaces T spanjf ) and @ span{g) are orthogonal, since JS < a. We summarize these results 
in terms of the principal subspaces: &(T,Q) - &{T,0) - @p(T,@) - 0{f,g) U 7t/2, where 
cos^ 0{f, g) = (J3{1- a)) I {a{ 1 -fS)), (the latter equality can be derived by elementary calculations, 
see [18, Theorem 7.2, p. 131]); spanjf) and spanjg) is one pair of principal subspaces and 
T spanjf) and Q spanjg) is the other, corresponding to the angle 7r/2. 

In multiplicative Schwarz DDM with an overlap for two subdomains, the error propagation 
of a simple iteration is given by (6. 1) and the convergence rate is determined by the quantity 

[(Pr^'f?)!^! = cos2^?(/,g) = {J3{\-a))l{a{\ -/?)) < 1, 

which approaches one when the overlap a - fi becomes small. At the same time, however, the 
CG method described, e.g., in [4, 38], converges at most in two iterations, since the spectrum of 
A in (6.2) consists of only two eigenvalues, 1 - cos^ 0{f,g) — sin^ 0(f,g) and 1. 

In the additive DDM the error is determined by (6.4) and the spectrum of A, the sum of two 
orthoprojectors, by analogy with Theorem 2. 18 consists of four eigenvalues, 

1 - cos 6(f,g) ^2sm\6(f,g)/2), 1,1+ cos 6(f,g) ^ 2cosHe(f,g)/2), and 2, 

therefore the CG method converges at most in four iterations. Similar results for a finite differ- 
ence discretization of the ID diffusion equation can be found in [12]. 
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